Papers by Tarek Al Mustafa
Enhancing Domain-Specific Encoder Models with LLM-Generated Data: How to Leverage Ontologies, and How to Do Without Them (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a new method for continual pretraining transformer encoder models is proposed for specialized domains with limited training data. |
| Approach: | They propose to use LLM-generated data to enrich domain-specific ontologies and pretrain transformer encoder models as an ontology-informed embedding model for concept definitions. |
| Outcome: | The proposed method improves on standard MLM pretraining on invasion biology domains. |